A  Note  on  Non-Linear  Approximation  Theory 


A.  A.  Goldstein 


D  D  C 

OCT  1  9  1%6  f 


TOE  7EDBRAL  9CBSNTIFIC  AND 
[  technical  information  __ 

“T7 


mathematics  research 


A  NOTE  ON  NON-LINEAR  APPROXIMATION  THEORY 


♦Supported 

Scientific 


by 

A.  A.  Goldstein* 

University  of  Washington,  Seattle 


Mathematical  Note  No.  482 
Mathematics  Research  Laboratory 
BOEING  SCIENTIFIC  RESEARCH  LABORATORIES 
September  1966 


>y  Air  Force  Office  of  Scientific  Research  and 
Research  Laboratories 


the  Boeing 


ABSTRACT 


Recently  J.  R.  Rice  [1]  initiated  a  geometrical  study 
of  non-linear  approximations.  In  what  follows  below  we  offe 
a  small  contribution  to  certain  analytical  aspects  of  mean- 
square  non-linear  approximations.  One  result  is  to  exhibit 
a  family  of  non-linear  topological  subspaces  of  the  space 
C[a,b]  with  the  mr an-square  metric  which  has  a  local  unique 
best  approximation  property. 


Recently  J.  R.  Rice  [1]  initiated  a  geometrical  study  of 
non-linear  approximations.  In  what  follows  below  we  offer  a  small 
contribution  to  certain  analytical  aspects  of  mean-square  non-linear 
approximations.  One  result  is  to  exhibit  a  family  of  non-linear 
topological  subspaces  of  the  space  C[a,b]  with  the  mean-square 
metric  which  has  a  local  unique  best  approximation  property. 

Let  S  be  a  subset  of  a  normed  linear  space  E.  Let  S  be 
any  open  set  containing  H(S),  the  convex  hull  of  S.  Let  A 

denote  a  map  from  S  into  a  real  inner  product  space  F.  Assume 

the  range  of  A  is  not  dense  in  F.  Assume  that  A  is  twice 
Gateaux  (G)  differentiable  on  H(S) .  Assume,  moreover,  that  for 
each  point  x  c  H(S)  these  G-derivatives  are  bounded  linear 
operators,  that  is  A'(x)  is  a  bounded  linear  operator  from  E 
to  F  and  A"(x)  is  a  bounded  linear  operator  from  E  to  the 
space  of  bounded  linear  operators  from  E  to  F.  Let  p  be  a 
point  in  F  which  is  not  in  the  closure  of  the  range  of  A.  Set 

h(x)  =  |  (  A  (x) -p  j  |  2 ,  B  =  sup { 1 1  A" (x) 1 1  :  x  c  H (S) } , 

y  =  inf  { ||  A*  (x)k  1 1 /  || k||  :  x  c  H(S)  and  x+k  e  s} , 

Y  =  inf  { ||  A(z+k)-A(z)  ||/||k||  :  z  and  z+k  c  s}  and 
C  =  sup { 1 1  A ' (x) 1 1  :  x  e  H ( S ) } .  The  notations  A'(x)k  and 

(A"(x)k)(u)  will  be  employed  for  first  and  second  differentials, 
respectively.  For  simplicity  the  norms  in  E  and  F  are  denoted 
by  the  same  symbol  1 1  •  1 1  . 
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We  show  first  that  if  z  c  S  is  stationary  point  of  h, 
that  is  h'(z)k  =  0  for  z+k  e  S,  and  h(z)  is  sufficiently 
small,  then  z  is  global  minimizer  for  h.  State  otherwise, 

A(z)  is  a  best  approximation  to  p  out  of  the  range  of  A. 

Moreover,  z  is  unique. 

Lemma  1 .  If  A  is  twice  differentiable  on  H(S) ,  then  so 
is  h.  Let  0  =  z  +  ok  for  any  o  e  (0,1),  where  z  and  z+k 
are  in  S.  Then 

W(0)k)(k)  =  [A(z)-p,(A"(0)k)(k)]  +  [A(0)-A(z),(A"(0)k)(k)] 

+  [A' (0)k,A'(e)k] . 

Proof .  If  f  :  E  -*■  F  is  Frechet  differentiable  and  g  :  F  ■+  R 
is  Gateaux  differentiable,  the  composition  gf  satisfies  the  chain-rule 
h' (z)k  =  g* (f (z) )f 1  (z)k.  (See  e.g.  [2,  p.  659]).  If  f ( 0 )  =  A(0)  -  p 
and  g(x)  =  1 1 x 1 1  ^ ,  then  h1  (0)k  =  2[A(0)-p,A' ( 0 ) k ] .  By  writing 
^  [  h' (0+tk)k-h'  (0)k]  «  £  {  [ A(0+tk)-A(0)+A(0)-p, 

(A1 (0+tk)-A* (0) )k+A'  (0)k]-[A(0)p,A’ ( 0 ) k ]  },  expanding,  and  passing  to 
the  limit  as  t  -*■  0,  we  get: 

is(h"(0)  (k)  =  [A(9)-p,(A"(e)k)(k)]  +  [  A*  (0)k,A'  (  0  )  k  ]  . 

Whence  the  lemma. 

Theorem  1.  Assume  that  C  and  B  are  finite  and  that  y  and  y 

are  positive.  Assume  z  is  a  stationary  point  for  h.  If 
9  or 

h(z)  <  y  /B(l  +  —),  then  h(z)  <  h(x)  for  all  x  ^  z  in  S. 

y 
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Proof .  Suppose  that  z  is  not  a  unique  best  approximation. 
Then  for  some  k  #  0  and  z+k  e  S,  1 1 A (z-t-k)  — p  1 1  <_  h (z )  .  Since 
]  |  A  (z )  — p  !  |  =  h(z)  it  follows  that  2h(z)  >_  |  |A(z+k)-A(z)  ||  >J|k||  y; 
whence  ||k|(  <_  2h(z)/y. 

Since  h'(z)k  =  0  for  all  k  c  E  we  have  by  Taylor's  Theorem 
that  h(z+k)  -  h(z)  =  (h" (0)k) (k) /2  for  some  8  on  the  open  line 
segment  joining  z  and  z+k.  It  would  be  a  contradiction  *:o  show 
h(z+k)-h(z)  >  0. 


By  the  above  lemma  we  have: 


W(0)k)(k)  >  -Bh(z)|!k||2  -  CB  [  |  k  1 1 3  +  y  2  |  |k  1 1 2 , 


where  the  inequality  ||  A(0)-A(z)  1 1  <_  C  [  |  k  [  |  has  been  employed 

[2,  p.  659].  Since  ||kjj  <_  2h(z)/y,  it  follows  that  (h"(0)k)(k) 
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is  positive  whenever  h(z)  <  y  /B(l  +  — ) .  Q.E.D. 


Remark  1.  The  hypothesis  that  y  is  positive  may  be  replaced 

by  the  hypothesis  that  S  is  bounded  by  a  sphere  of  radius 
2  2 

R  <  y  /2CB.  Then  if  h(z)  <  y  /2B,  the  above  theorem  holds.  Observe 
also  that  if  A  is  a  linear  operator,  B  =  0  and  z  is  a  best 
approximation  independently  of  the  value  of  h(z). 


Remark  2 .  Assume  the  hypothesis  of  Theorem  1.  Assume  E  is 
finite  dimensional  and  A  is  defined  on  E.  Assume  that  for  some 
Xq  c  S  the  level  set  D  =  {x  c  E  :  h(x)  <_  h(Xg)}  is  a  proper  subset 
of  S.  Then  p  has  a  best  approximation  out  of  the  range  of  A. 


-4- 


Proof .  Take  x  in  D.  Then  h(Xg)  >_  y  ||  x-Xq  ||  -  h(Xg). 

Hence  D  is  bounded.  By  the  continuity  of  h,  D  is  closed. 

Clearly  in  seeking  the  minimum  of  h  we  may  confine  our  attention 
to  D.  Thus  h  achieves  a  minimum  on  D.  Q.E.D. 

Remark  3.  Observe  that  if  we  assume  the  hypotheses  of 
Remark  2,  and  the  additional  hypotheses  that  x^  itself  is  not  a 
minimizing  point  for  h,  then  for  every  p  e  F  there  exists 
stationary  points  x  for  h,  that  is  points  x  e  S  such  that 
[A(x)-p,A' (x)k]  *  0  for  all  x+k  e  S.  To  see  this  observe  that 
because  h  is  continuous,  D  has  interior,  and  therefore  a 
necessary  condition  that  a  point  z  minimize  h  is  that  h'(z)  *=  0. 

Remark  4.  If  S  is  complete  and  y  and  y  are  positive, 
then  the  ranges  of  A,  and  A' (x)  are  closed  for  all  x  e  H(S). 

Proof .  Let  {a(x^)}  be  any  Cauchy  sequence  in  the  range  of 
A.  Since  ||  A(x^)-A(xS)  j |  >_  y  |)  x^-xS  ||  ,  the  sequence  {x^}  is 
also  Cauchy  with  limit,  say,  y.  But  ||A(x^)-A(y)  ||  <_  C  ||  x^-y||  , 
showing  that  A  is  closed.  The  same  proof  works  for  A'(x). 

We  now  turn  our  attention  to  the  "reverse  problem"  of  approximation 
theory:  Given  a  point  y  on  the  range  of  A,  does  there  exist  a  point 
p  ^  y  whose  best  approximation  is  y? 

Theorem  2.  Assume  the  hypotheses  of  Theorem  1 .  Assume  that  S  -  E 
and  E  is  complete.  Fix  x  c  S.  There  exists  a  point  p  c  F  ~  range  A 
such  that  A  lx)  is  a  unique  best  approximation  to  p  if  and  only  if 
A'  (x)  is  not  onto  F. 
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Proof .  We  prove  first  that  the  not  onto  property  for  A'(x) 

is  necessary.  Otherwise,  | J  A (x)  — p  | j  <  ||  A(y)-n  ||  for  all  y  ^  x 

but  the  range  of  A' (x)  is  onto.  Since  [A(x)-p,A' (x)k]  =  0  for 

all  k  e  E,  and  we  may  choose  k  such  that  A'(x)k  =  A(x)-p  #  0, 

we  get  [A(x)-p , A* (x)k]  >0,  a  contradiction.  Conversely,  if 

A' (x)  is  not  onto,  then  since  the  range  of  A'(x)  is  closed,  it 

follows  the  deficiency  of  the  range  of  A'(x)  exceeds  0.  Let 

0(x)  denote  the  orthogonal  complement  of  the  range  of  A'(x). 

Clearly  if  p  e  0(x)  +  A(x) ,  x  is  a  stationary  point.  By  Theorem  1, 
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moreover-,  if  ||  A(x)-p  ||  <  p  /B ( 1  +  — ) ,  then  A(x)  is  a  unique 
point  on  the  range  of  A  closest  to  p. 

Example .  Let  C[a,b]  denote  the  space  of  continuous  function 

rb 

on  [a,b].  In  C[a,b]  define  an  inner  product  <f,g>  *  /  f(t)g(t)dt. 

a 

In  E^  denote  by  [x,y]  the  inner  product  of  n-tuples  x  and  y. 

Let  ||  •  H2  and  ||*|l»  respectively,  be  the  norms  in  C[a,b]  and 
E^  which  arise  from  these  inner  products.  Let  v(t)  *  (v^(t) , . . . ,v  (t))  , 
v^  c  C[a,b],  (i=l,2, . . . ,n) ,  be  a  Haar  family  on  [a,b].  This  means 
that  for  any  subset  of  distinct  points  t^,  (1  <  i  <  n) ,  the  matrix 
{vj(t^)  :  1  <  i  <  n,  1  <  i  <  n}  has  rank  n.  Thus  if  multiple  roots 
are  counted  only  once,  the  generalized  polynomial  p(x,t)  =  [v(t),x] 
has  at  most  n  -  1  roots.  Let  P  =  ||  p(x,*)  j|  ,  i.e. 

II  [v(*),x]  ||2  £P||x  ||  for  all  x  e  En- 

We  now  specify  the  mapping  A  discussed  above  as  follows: 

A(x)  =  f([v(*),x]),  where  f  is  the  continuous  real-valued  function 
specified  below.  Thus  A  :  E  **  C[a,b]. 
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k 


*  t 


Specifically,  let  f  be  a  differentiable,  real-valued  function 
defined  everywhere  on  the  reals,  R.  Assume  that  f 1  ( t )  >_  a  >  0  for 
all  t  e  R,  and  that  f"  exists  and  is  bounded  above  on  R,  say 
by  N. 

Lemma  2.  The  mapping  A  satisfies  the  hypotheses  of  the  above 
Theorems  1  and  2. 

Proof.  Because  x  ^  0  implies  [v(*)»x]  has  at  most  n  -  1 

roots,  1 1  [  v(  •)  ,x]  ||2  >  0  for  all  x  4  0.  Thus  1 1  [  v  ( • )  ,x  (|2  >_  B||x|| 

for  some  B  >  0  and  all  x  e  E  .  We  shall  denote  the  point  A(x) 

n 

in  C[a,b]  by  A(x,*)  when  the  pointwise  values  A(x,t)  are  singled 
out  for  attention. 

Since  A'(x,t)k  =  f ' ( [v(t) ,x] ) [v(t) ,k] ,  we  get  1 1 A' (x, *)k  II 2  * 

II  f’([v(-),x])[v(*),kl  |j2  >_  ot  1 1  [v(  •)  ,k  1 1 2  >_  otB  |  |k  1 1  =  p||k||.  Similarly 
using  the  ordinary  mean-value  theorem  on  the  real-valued  functions 
f ( [ v (t )  ,  • ] )  defined  on  the  ray  {z  *  x+0k  :  0  <  0  <  1 }  we  get 
||  A(x+k,  *)-A(x,  •)  ||2/  1 1  k  1 1  =  ||f([v(')>x+k])  -  f  ( [v(  •)  ,x] )  1 1 2/  |  |k  1 1  - 

II  f '  ([v(0  ,x+0k])[v(*)  ,  ]  |j2  >_  a  1 1  [  v(  • )  ,  u^jj-  )  ||  2  =  p.  Thus 

Y  *  p  in  the  above  Theorem  1. 

By  Remarks  2  and  3  there  exists  at  least  one  point  z  e  S  such 

that  [A(z)-p , A' (z)k]  =  0  for  all  k  e  E  . 

n 

We  calculate  that  1 1  (A"(x)k)  (k)  ||  * 

||f"([v(*),x])[v(»),k][v(»)k]  |J2  <_  NP2  1 1  k  1 1  2 ,  whence  NP2  »  B.  Let 
M  =  max{f'(r)  :  r  e  Rq}-  Then  1 1 A  *  (x)  1 1  <_  MP  ■  C. 


It  remains  only  to  check  that  A  and  A'(x)  are  not  onto  F. 
The  former  follows  because  f  is  strictly  monotone  and  continuous. 
Thus  the  range  of  A  is  a  homeomorphic  image  of  the  span  of 
|vp...,v  },  which  is  a  finite  dimensional  subspace  of  C[a,b]. 

For  the  latter  we  observe  that  the  range  of  A'(x)  for  each  x  is 
spanned  by  the  functions  f '  [  (v(  *)  ,x)]Vj  ,  1  <_  j  <_  n.  Thus  A’ (x) 
is  a  finite  dimensional  linear  subspace.  Q.E.D. 

Remark  5.  We  have  constructed  for  each  function  f  satisfying 
the  above  hypotheses  a  topological  subspace  of  C[a,b]  with  the 
mean-square  metric  which  is  homeomorphic  to  a  Haar  subspace.  This 
subspace  has  a  local  unique  approximation  property — each  point 
sufficiently  close  to  the  subspace  has  a  unique  best  approximation 
in  the  subspace.  Moreover,  every  point  on  the  subspace  is  the  unique 
best  approximation  to  some  nearby  point  which  is  not  in  the  subspace. 
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